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Abstract — We investigate the detection dynamics of a soft 
parallel interference canceller (soft-PIC), which includes a hard- 
PIC as a special case, for code-division multiple-access (CDMA) 
multiuser detection, applied to a randomly spread, fully syn- 
chronous base-band uncoded CDMA channel model with ad- 
ditive white Gaussian noise under perfect power control in 
the large-system limit. We analyze the detection dynamics of 
some iterative detectors, namely soft-PIC, the Onsager-reaction- 
cancelling parallel interference canceller (ORC-PIC) and the 
belief-propagation-based detector (BP-based detector), by the 
generating functional analysis (GFA). The GFA allows us to study 
the asymptotic behavior of the dynamics in the infinitely large 
system without assuming the independence of messages. We study 
the detection dynamics and the stationary estimates of an iterative 
algorithm. 

We also show the decoupling principle in iterative multiuser 
detection algorithms in the large-system limit. For a generic 
iterative multiuser detection algorithm with binary input, it is 
shown that the multiuser channel is equivalent to a bank of 
independent single-user additive non-Gaussian channels, whose 
signal-to-noise ratio degrades due to both the multiple-access 
interference and the Onsager reaction, at each stage of the 
algorithm. If an algorithm cancels the Onsager reaction, the 
equivalent single-user channels coincide with an additive white 
Gaussian noise channel. We also discuss ORC-PIC and the BP- 
based detector. 

Index Terms — generating functional analysis, Code-Division 
Multiple-Access, iterative algorithms, detection dynamics 

I. Introduction 

DETECTION DYNAMICS of generic iterative CDMA 
multiuser detectors, which utilize soft-decision in the 
large system limit, is presented in this paper The CDMA is 
a digital modulation system that employs spreading codes to 
enable access to a mobile communication system by multiple 
users [491, l!50l. The statistical-mechanical approach has been 
applied to evaluate the performance of various wireless com- 
munication systems ll46l . This kind of systems is widely used 
in communications and signal processing, such as the code- 
division multiple-access, the multiple-input multiple-output 
channels 1341 . ifTTll . and compressed sensing ||5l, ll42l . ifTSl . 

Various types of multiuser detectors utilizing soft-decision 
have been proposed so far ||50l, (|49], Ull, JzUD Tanaka 
has first evaluated the properties of the maximum a pos- 
teriori detector and the marginal-posterior-mode detector by 
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the statistical-mechanical analysis, which is called the replica 
method |44|, 1461 . l38l . The replica method has widely been 
applied to analyze communication systems ITTl . ITSl . l35l 
and other information theoretic problems. It can treat the 
properties of detection results, but cannot directly treat the 
detection dynamics of detectors. Since the optimal marginal- 
posterior-mode detector itself is known to be NP-hard ll50l . 
it is important to construct suboptimal methods. Iterative 
algorithms are generally useful as such methods from the 
viewpoint of computational cost. Various kinds of iterative 
detection algorithms have been developed to date f36l, fT^I, 
l22l . IH, EOl, El, |20|D The analysis of iterative multiuser 
detection algorithms is therefore expected to play an important 
role in developments and improvements of detectors. 

Recently, the state evolution to evaluate the dynamics of 
the approximate belief propagation has been proposed by 
Bayati and Montanari |2|. The detection dynamics of iterative 
algorithms which are characterized by a dense graph has 
attracted a great deal of attention from theoretical and practical 
viewpoints up to now f22)\, f47l, fl]. Kabashima has proposed 
the belief-propagation-based detector and analyzed its perfor- 
mance I22I . Tanaka and Okada have analyzed the detection dy- 
namics pTl of the soft parallel interference canceller proposed 
by Divsalar et al L12| by means of a dynamical theory for 
the Hopfield model l39l . Bayati and Montanari have analyzed 
the reconstruction dynamics of approximate belief propagation 
algorithm for compressed sensing [2 |. These existing studies 
I22I . BtI . m have succeeded in analyzing of various kinds of 
iterative algorithms including belief-propagation-based meth- 
ods. 

However, these analyses, such as density evolution [41], 
l47l and state evolution [2J, are justified only for the cases that 
the correlation between present estimates and their past values 
can be neglected. In other words, these can only be applied to 
the case where there is not a retarded self-interaction, which 
is caused by iterations and this is often called the Onsager 
reaction, and their predictions systematically deviate from 
computer simulation results in general. 

We have already applied the generating functional analysis 
to the hard-PIC, i.e., Varanasi's conventional PIC E?], ||28l 
so far. This analysis can however treat the hard-decision only 
I27I . l28l . The GFA, which uses the saddle-point method 
im, I25I . allows us to study the asymptotic behavior of the 
dynamics in the infinitely large system IJl j . I16J, 17J. Since 
it is not based on the S/N analysis, it does not therefore 
need the Gaussian assumption of the noise part. In the S/N 
analysis, the signal part, that contains the user's information 
being estimated, is separated from the remaining noise part; 
besides the noise part is generally assumed to follow a given 
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distribution such as the Gaussian distribution. 

In this paper, we investigate the detection dynamics of some 
iterative algorithms for CDMA multiuser detection, applied 
to a randomly spread, fully synchronous base-band uncoded 
CDMA channel model with additive white Gaussian noise un- 
der perfect power control. We here treat soft-PIC, the Onsager- 
reaction-cancelling parallel interference canceller, which is 
an analogue of soft-PIC, and the belief-propagation-based 
detector. These models have the retarded self-interaction. 
To confirm the validity of our analysis, we have performed 
computer simulations under some typical system loads and 
channel noise conditions. 

This paper is organized as follows. The next section ex- 
plains the system model. Sections III and IV introduce some 
multiuser detection algorithms and the generating functional 
analysis, respectively. Sections V - VII present analyses of 
soft-PIC, ORC-PIC and the BP-based detector Section VIII 
explains the stationary states of iterative algorithms and their 
stability. In section IX, we discuss the decoupling principle 
lH?!, fTSl in iterative detectors. The final section is devoted to 
a summary. 

II. System Model and Notations 

Let us focus on the basic fully synchronous A'-user base- 
band direct-sequence / binary phase-shift-keying CDMA chan- 
nel model with perfect power control as 



1 



K 



(1) 



where 



k=l 

is the received signal at chip 



interval G 

e {-1, 1} are 



{1, • • • , N}, and where bk G {—1, 1} and s 
the binary phase-shift-keying-modulated information bit and 
the spreading code of user k e {I,-'' i^} chip interval 
/I, respectively. The Gaussian random variable aon^, where 
^ Af{0, 1), represents channel noise whose variance is 
ctq. The spreading codes are independently generated from the 
identical symmetric distribution P(s^ = 1) = P{s'^ — —1) — 
1/2. The factor 1/\/N is introduced in order to normaUze the 
power per symbol to 1. The signal-to-noise ratio is obtained 
as Eij/Nq ~ 1/(2(Tq) by using these normalizations. The ratio 
(3 — K/N is called system load. 

In this paper, the letters fc, k' denote indices in {1, • • • , K} 
and the letters /i, /i' denote indices in {1, • • • , N}. The (fc, k') 
element of the matrix W, whose index is a pair of the 
user numbers, is indicated as Wkk' ■ The elements of vectors 
y = (yi? ■ ■ ■ : Un) and b — (foi, • • • , 6/f ), whose indices are 
the chip interval index or the user number, are indicated as 



and 

-1) . 



bk, respectively. The s-th element of vector x 



{x''^ , x'^^^ , ■ ■ ■ , a;*-*^) ' , whose index is the stage number, is 
indicated by (x)^*', e.g., (x)^^^ = x^^^ (Note that (a;)^^) does 
not mean a;^~^^). Here, denotes the transpose of X. The 
(s, s') element of the matrix 

1-1) ••• 



X 



whose index (s, s') is a pair of the stage numbers, is indicated 
as e.g., = X(i'i) (Note that (X)(i'i) 

does not mean X*^^^'^^^). The notations are summarized in 
Appendix [A) 

III. Detection Algorithms 

We discuss the detection dynamics of the following three 
kinds of iterative detection algorithms in this paper 

A. Soft-PIC 

The soft parallel interference canceller has been proposed by 
Divsalar et al 1 12 |. Kaiser and Hagenauer have also proposed 
a similar algorithm ll24l . 

Definition 1: (Soft-PIC) The updating rule for tentative 



decision Vi^ G M of bit signal bk at stage t is 



K 

f[hk- y, Wkk'bt, 



k' = l,^k 



(2) 



where / : M — > M, which is called a transfer function, that is 
arbitrary and hk is the output of the matched filter for user k: 



1 ^ 



(3) 



and Wkk' is the fcfc'-element of sample correlation matrix W 
of the spreading code: 



1 



N 



Wkk' 2^3^81,. 



(4) 



The initial condition of iteration is b\, = 0. When the 
transfer function chooses f{x) = tanh(a;/cr^), this iterative 
detection algorithm is called soft-PIC. Here, is a control 
parameter representing the detector's estimate of channel noise 
variance. □ 
Miiller and Huber have improved Soft-PIC and have numer- 
ically evaluated its performance 1,33 J . 

B. ORC-PIC 

In soft-PIC, matched filter output has a very complex cor- 
relation between all estimates. The correlation due to iterative 
calculation worsens performance of detection. The ORC-PIC 
is an analogue of soft-PIC, which has a term to cancel such 
correlation. The updating rule is modified to 



b^k^ -fihk-Y^Wkk'b^k' 



(t-l) 



k'^k 



^(i.-l)J,(-l) 



(5) 



with b 



(-1) 



= 0. If we properly choose coefficients {f ■*} of 

the term -f ^k'~^%~^^^ the correlation 

can be cancelled. Tanaka and Okada derived the preceding 
parameter {f •*} by applying density evolution 1,47 J based 
on the statistical neurodynamics 
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Definition 2: The ORC-PIC is defined by updating rule Q 
with 



(6) 



and (rfe6fe)(-i) = (ffe6fc)^°^ = 0. Here, b^*^ = ••• , 

fc^*-*)^, Ffe is a (t+1) X (< + l) matrix whose (s, s') element is 
f and G*-*'*^^-* denotes the average single-user response 
function. □ 
Detail of the parameter G*^' *"^-' is introduced later (in Section 
IV-Bt . Note that G^*'*~^^ does not depend on the user index k. 



C. BP-based Detector 

Assuming that information bits b are independently gen- 
erated from the symmetric distribution, the posterior dis- 
tribution from received signals y is given as p{b\y) — 
P(y|b)/Ebe{±i}^P(y|^)' where 



N 



2Tra 



(7) 



and a is a control parameter when true noise level parameter 
(To is not known. The marginal-posterior-mode detector 
is represented by 



bk = argmax p{b\y) 



(8) 



The BP-based detector is an iterative algorithm that employs 
the belief propagation to approximately calculate the posterior 
marginal included in (O. 

Definition 3: (BP-based detector) The BP-based detector 
is given by the following iterative equations: 

= tanh(i?W/i - J7(*) + ^WfoW), (9) 
[/(*) = ^(*)M^6(*) + - gW)[/(*~i\ (10) 



where 



rW ^ yl(') + - QW)i?(*-i\ (11) 

1 



a2+/?(l-Q(*))' 



)(*) ^ i.^^^(*)^2 
fc=i 



(12) 
(13) 



with initinal conditions: i?*^^^^ = A^"^'^ and 6^ — 0. Func- 
tion tanh is applied componentwise. From posterior average 
6l*\ the tentative decision at the t^^ update is evaluated as 
bj^^ = sgn(6{.*''), where function sgn (x) denotes the sign 
function taking 1 for a; > and —1 for x < 0. □ 
This BP-based detector can be rewritten as 

6i*+^'=tanhfi?W/i,+.4W6i*) 



K 



k' = l 



(14) 



where 



'0, s = -l 

Ai^) Yl A'^''^ I3[l ~ q^''^], 0<s<t-l 



AW, 



(15) 



IV. Generating Functional Analysis 

In this section, we briefly summarize methods in GFA. The 
generating functional analysis, or the path integral methods, 
have been applied to the model which is described using 
realizations of random variables by de Dominicis [ 11 1. In GFA, 
one can analyze the asymptotic behavior of the dynamics in the 
infinitely large system using the generating functional which 
can be regarded as a kind of the characteristic function in 
statistics. 

We here consider the following model. Let vectors a;^''' = 



pK 



be the states of the model at stage s and let the 



initial state be x*^ ^\ Let the updating rule be 



(16) 
denotes a 



for s e {-1, 0, • • • , t - 1}, where T : R - 
function and w denotes parameters in it. 

If we know the probability of seeing a particular state 
at a given stage ps{x'^^^), we can evaluate the property of 
the system by using it. In GFA we consider the probability 
of observing a particular sequence or path of states, i.e., 
p{x^~^\x'^'^\ ■ ■ ■ ,a;(*') up to some finite time t instead of 
Ps{x^'^'>). The way to do this is to introduce a generating 
functional which is defined as 



exp 



-1 fc=i 



(17) 



where the bracket 
probability p{x^^^'' 
the dummy variables ■0 



(•) denotes the average over the path 
x^'^^ ■ ■ ■ ,xW) and we have introduced 



Taking derivatives 



with respect to the dummy variables aUow us to examine some 
averages, e.g.. 



= i lim 



= — lim 



di:\!'^dA' 



(18) 



(19) 



It would not be possible to calculate the generating functional 
for a particular realization. We thus assume that the gener- 
ating functional is concentrated to its average over random 
variables w and calculate 'K^{Z[ip]), where the Ej^ denotes 
the expectation with respect to random variables X. 

Averaging over the random variables, we will move to a 
saddle-point problem in the large system limit K oo. The 
terms in the averaged generating functional can be split into 
three related parts. The first one is a signal part. The second 
one is a static noise part due to the random variables within 
the model. The last one is retarded self-interaction due to the 
influence of the state at the previous stage, which may be able 
to affect the present state. The GFA allows us to treat the last 
part. 
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V. Generating Functional Analysis for Soft-PIC tentative decisions as 



A. Averaged Generating Functional 

The goal of multiuser detection is to simultaneously infer in- 
formation bits &i, • • • ,bK after base-band signals yi, - ■ ■ ,yN 
are received. 

If the transfer function takes f{x) = sgn (x), this updating 
rule coincides with the hard-PIC (49). Note that the hard-PIC, 
which means the Varanasi's conventional PIC, is also obtained 
by taking the limit cr ^ in soft-PIC. 

We assume the matched filter stage, i.e., 6^(0) = f{hk), for 
initialization. This initialization is easily treated by formally 
assuming ~ S(b[. for all k, where S denotes the 

Dirac delta function. The widely used measure to determine 
the performance of a demodulator is the bit error rate (BER). 

sgn(&^*^) at the t^^ 



The BER of hard decisions fe^*-* 



stage of soft PIC is given by P^^''' = (1 — m)_^')/2, where 



m 



^ — -f^ J2k=i^kb'j^^ is the overlap between information 
vector 6^*' = • • • , Bk)^ and tentative hard-decision vector 
(bf\--- ,b^^)'^. Without loss of generality, we can assume 
that the true information bits are all 1, i.e., bk = I for all k, 
because the spreading codes are symmetric. 

Let us analyze the detection dynamics in the large system 
Umit where K, N ^ oo, while the system load /3 is kept finite. 
We introduce inverse temperature 7 for generating functional 
analysis. The stochastic updating rule for tentative decision 
fol*' G M of bit signal bk at stage t is given by 



7 



27r 



with 



,(*) 



= hi. - 



K 

E 



Wuu'b^l) 



(20) 



(21) 



which is a summation over all messages from other tentative 
decisions. Note that this updating rule coincides with b[f = 
f{u\^^)+M{0, 7^'^). In the limit where 7 — s- 00, this stochastic 
updating rule is equivalent to the deterministic rule (|2). Term 
is a stage-dependent external message that is introduced 
to define a response function. The inverse temperature and the 
external message are respectively set to 7 — > cx) and 6*^*' = 
at the end of analysis. 

The stochastic updating rule for tentative decision vector 
b(*) = (5^*)) g at stage t is given by using ( |20l ), 

i.e., = nf=iP(^i''^^^l^^'^)- The dynamics is 

a Markov chain, since the present tentative decision depends 
only on the past decisions. A path probability (density) 
p{b^^^\ • • • , b*-*') is therefore simply given by the individual 
transition probability jb'*)) of the chain: 



[ (f[duApiU~'\---,U'y)g. (23) 



To analyze the detection dynamics of the system, we define 
a generating functional as ^[-0] ^ (exp[— i X]s=-i ^^^'^ ' 
i/jf'^)]), where b(^) = {b["\ ••• , and rp^'^ = 

• • • , V'if^)^- The basic idea underlying generating functional 
formalism is very simple ifTTl . O, Q. If the generating 
functional Z[if>] can be evaluated as a functional with respect 
to dummy functions one can obtain all averages of in- 

terest by differentiation from Z[ip]. The generating functional 
includes the random variables {s^} and {n''}. 

We here assume that the generating functional is self- 
averaging, namely, in the large system limit, the generating 
functional is concentrated to its average over random variables 
{s^} and {n^} and the typical behavior of the system only de- 
pends on the statistical properties of the random variables. We 
therefore evaluate the averaged generating functional defined 
as follows. 

Definition 4: (Average Generating functional) The average 
generating functional ^[■0] is defined by 



Z[ip] = ( exp 



-i ^ b(^) • 



(24) 



where [• • • ] = ^si,--- ,SK-n[' ' '] denotes the average over 
spreading codes Si, • • • , sk and the noise n. □ 
From the averaged generating functional, we can obtain 



lim^ = - 



k I 



lim , , , 
hm j—^ — z-^i- 



del: 



Calculating the average over spreading codes Si, 
the noise n, we have the following result. 



(25) 
(26) 

(27) 
, Sk and 



Lemma 1: The averaged generating functional is simplified 



to 



Z[xl}] = / dridf]dkAkdqAqdQdQdLdL 



X exp 



K{<i> + -^ + n) + 0{\nK) 



, (28) 



in which functions $, and Vl are given by 



3(b(- 



,6(*)) =p(b(-i)) W piU'+'^'^lU'^). (22) 



The initial state probability becomes p(6'^~^^) = 
rifcLi ^(^fc ^^)- Therefore, we can calculate an expectation 
with respect to arbitrary function Q = G{b'^~'^\ ■ ■ ■ of 



s=-l 

t-1 t-1 



{s.s') (s,s') 
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K 



k=l 



R'+2 



X exp 



s=-l s'=~l 
s=-l 

s=-l s=-l -I ^ 



(30) 



of the averaged generating functional: 



lim 



lim , , , 
lim J—. j-p- 



(32) 

^-{l-Sk,k'){b^'^)k{b^'"^)k' 

~Sk,k'{b^'^b^''^}k, (33) 



lim 

lim r—, ;r77- 

'/'^o del"' del', 



{l-Sk,k'){b^'^)k{u^'"^)k' 

~Sk,k'{b^'^u^''^)k, (34) 
(35) 



0, (36) 



17 = — In / dvdvdwdw 
K 



r AT t-1 



X exp 



i E E ^^^'^ + 4'^^!.'^ - 

/.i^l s^— 1 



N t-1 t-1 

^EE E«W^+^ 

fl—l S — — 1 s'— — l 






^ N t-1 t-1 

^EE m'^'-' - 

fl—l S — —1 s'— — l 












^ N t-1 t-1 

^EE EH/^ 

/J.— 1 S — —1 s'— — l 










(31) 



□ 

Details on the derivation and definitions of the notations are 
given in Appendix 151 



B. Saddle-Point Equations and Meaning of Macroscopic Pa- 
rameters 



One can deduce the meaning of macroscopic parameters 
by differentiating the averaged generating functional Z[xl)] of 
(l28T l with respect to the external messages 6*^"^ and dummy 
functions The averaged generating functional ^[■0] is 

dominated by a saddle-point for K — > oo. We can thus 
simplify the saddle-point problem in Lemma [T| in the large 
system limit. Using Z[0] = (1) = 1, we now find derivatives 



where ( )k denotes the average as 



W dUA I dMduwk{{b,u,u})f{{b,u,u}) 
W d6("A f duduwk{{b,u,u}) 

5 = -l ^ 



(37) 



with 



Wki{b,u,u}) 
= 6[b'--^^]exp 



E{i-4^-y[^^^^'^-/(-^^^)]'} 

S— 1 ^ 



s=0 s'=0 



s=0 

t-1 



s=0 



(38) 



saddle 



which is referred to as a single-user measure. Here, evaluation 
/[saddle dcuotcs an evaluation of function / at the dominating 
saddle-point. To derive the in (l35T l and in (|36] |. we 
use identities 



limMj^^MM.o, 
lim ^^^[^] _ 9^^[0] 



= 0, 



(39) 
(40) 
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respectively. Substituting (|25]l - ^} and ^ - ^ into (|32] i 
- ( [34] i. we have 



(41) 



+a-5k.k'){b^'^)k{b^'">)k', (42) 



(43) 



In the large system limit K ^ oo, integral (I28t will 
be evaluated by the dominating saddle-point of exponent 
$ + ^ + ri. We can now derive the saddle-point equations 
by differentiation with respect to integral variables t]*^"', f}'^^\ 
k'^'K Q^'^^'^ Qt"^'''), and Lt"^'''). 

These equations will involve the average overlap m^*^ (which 
measures the bit error rate), the average single-user correlation 
C'-''"'^ and the average single-user response function G''*'^ 



K—^oo A 



k=l 
K 



K^oo K ^ 



(44) 
(45) 
(46) 



Using the identities (l32]l - (O and (|4T]i - (gSll, the straight- 
forward differentiation of $ + \E' + ri with respect to 77'^''), f]'^^\ 
k^'\ U'\ q^^'^'K g^'*'"'), Q^'*'"'), Q^^'^'K L^^'^'K and L^"'"'^ 
leads us to the following saddle-point equations; 



r}(«) = i 



saddle 
K 



lim 

A'-i-oo K 

. on 



k=l 



= 0, 



9r2 



saddle 



saddle 
K 



K ^ 



k=\ 



Q 



{s,s') _ 



saddle 



on 



lim — V(6( 



saddle 



(47) 

(48) 

(49) 
(50) 
(51) 

(52) 

(53) 

(54) 
(55) 

(56) 



for all s and s', respectively. Comparing them with (|44] | 



(|46] l. the following relationships are obtained: 



(57) 
(58) 
(59) 



Therefore, we hereafter make use of {m^'*\ C'-'*'* \ iG^''''' ■*} 
instead of {ri'^^\q'^^''^ \ L'^^''^ )}. It should be noted that causal- 
ity 



0, 



(60) 



should hold for s < s', therefore i^'*^*') = G^^ = for 
s < s'. 

C. Derivation of Saddle-Point Equations 

The integral in with respect to v and w is calculated as 
^ 1 /■ dv dw 



P J (27r)(*+i)/2 (27r)(*+i)/2 



xe 



dt) 



PJ (27r)(*+i)/2 

Xe"5'>^Q*|_D|-l/2g-i*^(/3-'l-B)^D-H/5-'l--B)« 

= _J_{ln|£)| 

+ ln |g + {p-H - BYD-\p-H - B)\}, (61) 
where B and £) are matrices whose elements are defined by 
A (62) 

2 

respectively. The 1 denotes an identity matrix. 

The saddle-point equations including are evaluated as 
follows. We find 



d 



saddle, fe=0,Q=O 

^ {in|r>| 



9m(^) V 2/3 
+ ln + GYb-\p-H + G)|} 



5 



9m(«) V P 



= 0, 



(64) 



and find q^^'* ^ = in the same way. Similarly, the L^^''^ ' 
becomes 



917 



d 

'd&^y p 



saddle, fe=0,Q=O 

^ ln|J7| 



(65) 
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where we put U = f3 ^1 + G, whose (s,s') element is 
jjis,,') A /^-ij^^, + G(^'"'\ and [/(^^"') denotes a cofac- 
tor of the (s, s') element of U. We therefore have L — 
_ 1 (jyT)-! = (1 - I3G'^)-^. Since Q = O, n can be ex- 
panded with respect to Q as In | A + Q| = tr In A + tr A^Q. 
The Q^'*''* •* can be evaluated as 



Q 



I lim -, — 77 

lim 



saddle, fc—O 



d 



2/3 Q^o 50(^^^') 
2/3 



(66) 



where we put 



D 



We then have Q = -i^M^ = -ii(l + /3G)-iD(l 
/3G'^)-^ Finally, we turn to 



(67) 
(68) 
(69) 



fc(^) =i lim ^, , 



d 



lim , 

/3 k^o dk(' 



saddle, Q=0 

.ln|l~/3BT| 



= Ar 



(70) 



where A[s] denotes a + 2) x (t + 2) matrix whose (s', s") 
element is given by 

1, for s' = s 



A 



(71) 



The A[s] is a matrix whose elements in a row that represents 
values of stage s, i.e., the (s + 2)th row, in 1+/3G are replaced 
to 1. Since |1 — I3B^\ contains fc*^*^ only in a single row, 
|1 — is expanded with respect to the row to derive =(°' 

in dTOll. 

Example 1: Parameter A[2] has the following form: 

/ 1 /3G("'-i) /3G(i'-i) /3G(2.-i) \ 
1 /3G(i'0) /3G(2^o) 
1 /3G(2^i) 

V 1 1 1 1 

when t — 2. 



(72) 



□ 



D. Bit Error Rate 



One can obtain the polynomial expressions of soft-bits 
b\^\ which are averaged over the path probability (l22t . by 
GFA. Therefore, we can also evaluate the averaged value of 
analytic functions with respect to the soft-bits. The n-th order 
differentiation of the averaged generating functional, which 



has the forms of (l24l l and 



d 



(s) 

with respect to ipl gives 



lim 1 , > 



(a) 



where =(°^ and =^'''> are derived by differentiation of (l24l 
and ( |28] l, repsectively. For arbitrary function F{x) that can be 
expanded around a; = 0, we thus have the identity: 



(74) 



The Hamming distance between single letters is defined by 

0, ii X ~ X 

1, if a: 7^ x ' 



d{x, x) 



(75) 



The Hamming distance between if-bit sequences b and 
b is measured by the averaged single-letter distortion as 
d{x,x) = K~^^^^^d{bk,bk)- The hard-decision estimates 
can be represented by b\^^ — sgn (fc^*'') If we therefore choose 
F{x) such as a function which approaches sgn (x), e.g., 
lima-i-oo tanh(aa;), the bit error rate P^^*-* can then be evaluated 
as 



(76) 



=^(i-;|E(«g"(^fe^))0' (77) 

^ fe=i ^ 

where d{b,b) = (1 - bb)/2 for b,b e {-1,1}. 



E. Effective Single-User Problem and Analytical Result 

We here summarize our calculation. Some macroscopic 
parameters are found to vanish in the saddle-point: k'-"^ = 
q{s,s ) _ Q Yhe remaining macroscopic parameters can 
all be expressed in terms of three observables: the average 
overlap m'^^\ the average single-user correlation G'^'*'* \ and 
the average single-user response function G*^*'^ \ which are 
defined by ( |44] | - ( |46] |. The averaged generating functional 
Z[il}] is dominated by a saddle-point for K 00. We can 
thus simplify the saddle-point problem of Lemma [T| in the 
large system limit. 

We derive a single-user saddle-point problem. Note that 
we remained the user index k to deduce the meaning of 
macroscopic parameters with respect to external messages 
{olf^} and dummy functions {tplf^}- We set these parameters 



as 



0. 



(78) 
(79) 



Consequently, the single-user measure (l38T l becomes user 
independent. We then arrive at the following proposition. 

Proposition 1: Setting 7 00 and O^'^^ — 0, the dynamics 
of soft-PIC can be obtained as the following equations in the 
large system limit, i.e., K ^ 00. 

m(^) = ((6(^))), (80) 
G(^.«') ^ (81) 
q(s,s') ^ {{U'\r-^v)'-''^)), (82) 

with the causality: G*^^'* ^ = for s < s'. The bit error rate 
of hard decisions { sgn [b[^^]} at the t^^ stage of soft-PIC Q 
is obtained by 



=(-i)"((rr>fc^ 



(73) 



P 



-(!-(( sgn(6(^))))). 



(83) 
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The effective path measure is given by 

{{9ib,v))) 



Jl _ /(fc(^) + + (r6)(^))], (84) 



where 



with 



A 



R=il + /3G^)-i£>(l + pG)-\ 
r = (l + /3G)-i/3G, 
k^'^ = IAmI. 



for s' = s. 



1, 



(85) 
(86) 
(87) 
(88) 



(89) 
(90) 



The terms (R^^vY"'' and (TbY"'' denote the s^^ element of 
the vector R~^v and Tb, respectively. □ 
Equations (l80ll-(l90ll entirely describe the dynamics of the 
system. Term (Fb)'*' in ( |84] | is called the Onsager reaction 
term. 

F. Numerical Analysis and Experiments 

To validate the results obtained here, we performed numer- 
ical experiments in an iV = 8,000 system. The system of 
N — 8,000 is too large for a practical system, but we are 
now concerned with the verification of the analytical result 
derived under the large system limit. 

Figure [Tta) plots the first few stages of the detection 
dynamics of the hard-PIC { f{x) — sgn (x)) and soft-PIC 
(/(x) — tanh(a;/CT^) with a = ctq) predicted by GFA and 
density evolution ||47] for Ei,/N^ = 8 [dB]. The detailed 
derivation of the density evolution analysis is available in the 
reference |47|. The system load is /3 = 0.5 < /3c, where /3c is 
the critical system load defined as the minimum system load at 
which the dynamics fail to converge to the replica-symmetric 
solution of the marginal-posterior-mode detector The critical 
load /3c of soft-PIC for Eb/No = 8 [dB] is about 0.6. The 
predictions of density evolution systematically deviate from 
computer simulation results at transients. 

Figure [Ttb) plots the first few stages of the detection 
dynamics of the hard-PIC and soft-PIC for E^/No = 8 [dB], 
predicted by GFA and density evolution with the system load 
of /3 = 0.7 > /3c- Oscillation of the detection dynamics was 
observed, when /3 > /3c. The density evolution results have 
residual deviations in Fig. [T] due to the lack of the Onsager 
reaction term and the assumption that the summation over all 
messages follows a Gaussian distribution. In particular, the 
deviation between the density evolution predictions and the 
simulation results becomes large when /3 > /3c. In contrast, 
GFA exhibits good consistency with the simulation results. 



simulation (hard-PiC) A 
GFA (hard-PiC) 

DE (hard-PiC) -"O" 

simuiation (soft-PiC) □ 

GFA (soft-PiC) -0- 

DE (soft-PiC) -O- 




0.01 



2 

stage t 
(a) 



simuiation (hard-PIC) 


A' 




GFA (hard-PIC) 






DE (hard-PIC) 




simuiation (soft-PIC) 


□ 




GFA (soft-PIC) 






DE (soft-PIC) 















0.26 
0.24 
0.22 

0.2 
0.18 
0.16 
0.14 
0.12 

0.1 



(b) 

Fig. L First few stages of detection dynamics of the hard-PIC (upper) 
and soft-PIC with f{x) = tanh(a;/(T^) with a = ctq (lower) predicted by 
GFA (soHd lines) and density evolution (dashed lines). Computer simulations 
(triangles and squares) were evaluated with A'' = 8,000 for E^/Nq = 8 
[dB]. The system loads were (a) /3 = 0.5 < /3c and (b) P = 0.7 > /3c, 
respectively. 



G. Derivation of Existing Results by Generating Functional 
Analysis 

If we put Tb = 0, viz., if we neglect the Onsager reaction 
term, the GFA recovers the density evolution framework based 
on the statistical neurodynamics |47|. 

The multiple integral including ( [84l l can be partially per- 
formed, when we put Tb = 0. Namely, m'^'^\ G*-" \ and P^^'*-' 
only depend on w^"^^) among v; and C^'^''^ ^ only depends 
on w^*^^) and v^'^ ^^K We here separate v into two sub- 
vectors: V — {vi,V2)^ . The correlation matrix R can then 
be represented as 



For example. 



Rii 



R 



Vi and 
)^ and 



Rii 
R21 



R12 
R22 



(91) 



Rii are chosen as Vi = 



^(s-l,s-l) 
^(s'-l,s-l) 



Jlis-l.s'-l) 



(92) 



to calculate dSTT ). For arbitrary function /(fi), the following 
identity holds: 



(93) 
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Since it turns out that the response function becomes zero Proposition 2: The dynamics of ORC-PIC is given by the 
except for G'^*'*~^\ the fc^^^ has a simple form. following average over the effective path measure: 



i9{b,v))) 



We then obtain the following result. The bit error rate Pb(t) 
of hard decisions bk{t) — sgn [uk(t — 1)] at the fth stage is 

Pb(t)-Q^==_ , (94) 



t-i 

X 

s=-\ 



where Q(z) = Dx denotes the error function and we here 

put Dz = dz(27r)~^/^e~^^/^ to simplify the notation. The ^^^l) 

rn^*' are to be evaluated with the following set of recursive All other parameters are identical to those in Proposition [l] □ 

equations: For ORC-PIC, the summation over all messages, which is 



fcW =l--^G("^'-i)fc('*-i), (95) 



the argument of /(•••) in (|5]l becomes 



g' with external messages {0^ -^l. The averaged generating func- 

£)i^'S) (— /SG'-'''^^^-'), (96) tional ^['j/'] is represented as 



\=-l T=X+1 



^(^+1) ^ /■ Dz/(fc(^-i) +zVi?(^-i'^-i)), (98) xexp 



t+i)-fc 
^s — — 1 ^ 

t 



s=Ofc=l ^ fc=l 



^^^^ + E r^rW'^y:4n^'~oi^^ 

s' — — 1 ^ /i— 1 

(103) 

The difference between ( |24] | and ( 1103b is only in the fourth 
+ term f i^'^'-'fei"'' in • • ) in (fT03] l. One can straight- 

forwardly show Proposition |2] in the same manner as the 
derivation of Proposition [T| The difference in the analytical 
^■(^("'-1) _|_ z\/_R(s-i,s'-i) results only appears in (l84t of Proposition [T] 

We will now consider how to choose matrix F = (r'"''* 
in the Onsager reaction canceling term to cancel the Onsager 
reaction term. 



+ u^J R^^'-^^^'-^) - R(^-^'^'-^)). (100) 



The initialization condition is D = D = a^+Z?, ^ 3. ^hen the Onsager reaction term (f6)(*) is 

' chosen as 

This result is identical to that of density evolution [|47|. In 



the derivation by means of density evolution, it is assumed (rb)(*) = y^(— ( Tf G'^^^'^-* ) 6'*^ (104) 
that the local field Uk{t) follows the Gaussian distribution V J--'- / 



s=0 



with mean Bt and covariance Ct t- Furthermore, the Onsager „ . - - t \ 

reaction term is ignored. The GFA, on the other hand, can for i > and (Tby-^) = (TbY°) = 0, then the Onsager 
treat the Onsager reaction term coiTectly. reaction term Tb is canceled viz., Tb - Tb = O, and the 

response functions become G'*^* ' = except for t = t' + 1. 

□ 

VI. Generating Functional Analysis for ORC-PIC Details of derivation is available in Appendix |C] Note that 

(|6]l is a recursive formula of (I104t . One can confirm the ORC- 
This section discusses the dynamics of ORC-PIC. pic algorithm can correctly cancel the Onsager reaction term 
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simulation (hard-ORC-PIC) A 

GFA (hard-ORC-PIC) -V" 
simulation (soft-ORC-PIC) □ 
GFA (soft-ORC-PIC) -0- 




matrix elements 



2 

Stage t 
(a) 



simulation (hard-ORC-PIC) A 

GFA (hard-ORC-PIC) 
simulation (soft-ORC-PIC) □ 
GFA (soft-ORC-PIC) 




0.26 
0.24 
0.22 

0-2 
I 0.18 
I 0.16 

X) 

0.14 

! 

0.12 
0.1 



(b) 

Fig. 2. First few stages of detection dynamics of the hard-ORC-PIC (upper) 
and soft-ORC-PIC with f{x) = tanh(a::/(T^) with a = uq (lower) predicted 
by GFA (solid lines). In this case, the density evolution result is identical to the 
GFA result. Computer simulations (triangles and squares) were evaluated with 
N = 8, 000 for Eb/No = 8 [dB]. The system loads were (a) /3 = 0.5 < /3c 
and (b) (9 = 0.7 > /3c, respectively. 



by GFA. The density evolution results completely agree with 
the GFA results [47 1. The density evolution curves of Fig. [T] 
are equivalent to the proformance of ORC-PIC. 

To validate the results obtained here, we performed nu- 
merical experiments in an = 8, 000 system. Figure 
Wia) plots the first few stages of the detection dynamics 
of the hard-ORC-PIC (f{x) = sgn (x)) and soft-ORC-PIC 
(/(x) = tanh{x / a'^) with a = cto) predicted by GFA |47| for 
Eb/No = 8 [dB]. Figure Hb) plots the first few stages of the 
detection dynamics of the hard-ORC-PIC and soft-ORC-PIC 
for Eh/No — 8 [dB], predicted by GFA and density evolution 
with the system load of (3 — 0.7 > f3c- Oscillation of the 
detection dynamics was observed, when /3 > (3c- 

VII. Generating Functional Analysis for 

BP-BASED detector 

We next apply this scheme to the BP-based detector One 
can obtain the following result in the same manner as soft-PIC. 

Proposition 4: The dynamics of the BP-based detector rep- 
resented by (fT4l l is described by equations (I80]l-(l84li and the 
following equations: {1- I3B)-^D{1 - /3B^)"\ T = 
diag(A(-i), ••• -El=-i Js+i(E>A^+HU^)-'V 

and fc*^*^ — \A[s]\ where B, U, and D are matrices having 



^(s..')=_ J2 (105) 

T = -l 

S 

= Ss^s' +P J^'^^'^G'^-''^ (106) 



T = -l 



r'=-l 
s s' 



T = -l 



_|_ ^ ^ j{s' ,t')^(t,t') 



T=-l t'=-1 



(107) 



and 



-13 J^"''^'^G^^'^''\ s'^s 



s = s. 



(108) 



and matrix Js denotes 

/ j(^-i'-i) 



j(i-l,t-s-l) 



V 



j(t-l,i-s-l) 






(109) 



Here, A = (^s+i^^/) denotes a (i + 1) x (t + l) matrix whose 
(s, s') element is given by (5s+i^ss and the operator (g) denotes 
the Hadamard product A (E) B = [aijhij) for A = [aij] and 
B = (fey). □ 
For the BP-based detector, the summation over all messages, 
which is the argument of tanh(- • • ) of (fT4l i becomes 

v^^R^^hu + A^Vt^- Y J^'-'^f2w,,,U:\ (110) 

s=-l k' = l 

The averaged generating functional Zlip] is represented as 



=E«„...,«,,,, 



du 



X exp 



/R(t-|-l)Jf 
^S — — 1 ^ 

-i J2 b^'^ ■ ^^'^ 

s=-l 
s=Ok=l ^ 
s=-l k' = l 



(111) 
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simulation (BP-based detector) 
GFA (BP-based detector) — 




Fig. 3. First few stages of detection dynamics of tlie BP-based detector 
predicted by GFA (solid lines). Computer simulations (tiiangles and squares) 
were evaluated with N = 8, 000 for E^,/Nq = 8 [dB]. The control parameter 
and the system load were set as ct = (tq ^nd /3 = 0.5. 



Details on the derivation are given in Appendix |D] For the 
first few time-steps, we confirmed explicit expressions for the 
solutions to our dynamic equations as 



(112) 



^ J Dztanh^{\/BM^z + k^^^), (113) 



^(t+i,t+i) 



a2 + I3[l - qit^t)] ' 
/3[1 - 2m(*) + q 



(cr2 + _ g(t,t)])2 



(114) 



(115) 



where the BER of hard decisions at the t stage is — 
with H{x) = Dz. In the BP-based de- 
tector, the Onsager reaction term vanished. These are identical 
to the results from density evolution |22|. Figure |3] plots the 
first few stages of the detection dynamics of the BP-based 
detector predicted by GFA [gTl for Eb/No = 8 [dB]. The 
control parameter correct was set as a = (Tq. 

On the other hand, Kabashima analyzed the stability of the 
density evolution results and obtained an unstable condition for 
the fixed point solution (22 \ . This unstable condition coincided 
with the Almeida-Thouless (AT) instability condition |1| of 
replica analysis obtained by Tanaka ll46l . The AT instability 
is the local instability of the RS saddle-point solution in the 
replica analysis. In GFA, it is known that the AT instability 
condition is derived by a condition that the response function 
diverges |431. When the AT instability condition is satisfied, 
the relaxation times of GFA dynamic equations are expected to 
extend to infinity. In the system addressed here, this condition 
must be equivalent to the AT instability condition. In Section 
IVIII-CI we will discuss the stability analysis that corresponds 
to the AT instability. 

VIII. Stationary Estimates 

This section explains how we approximately extract the 
stationary estimate, which is a stationary state of the iterative 
algorithm, under some assumptions. To simplify the problem, 
we restrict ourselves to the soft-PIC algorithm. We follow the 
method of Ref. Q. 



A. Analysis of Stationary Estimates 

We recognize that the representation of the effective path 
measure given by ( l84l i is fully equivalent to the measure 
corresponding to a single-user process of the form: 

Variable v — (w*^*^) can be regarded as a random variable 
that obeys Normal distribution Af{0, R), e.g., {v^^^)^ — and 
= from (ISlll, where (•••)„ denotes the 

average over random variable v. We here put 

^(t+l) A fc(t) ^_ + g(t) _^ (r^)(*), (117) 

then the single-user process can be rewritten as 6'*-* = /(i/'^*-'). 
We therefore have the following form. 



.(*+!) = m + + + r(*'*')/((/.(*')), (118) 

t' = -l 



The relationship of (II 18l l is referred as a single-user process. 
Using ( II 18b . the overlap (ISOl l. the correlation funciton (|82] |. the 
response function (|82] |. and the bit error rate (i8Jl are obtained 

as 

= (119) 

= (/(0(*))/(</>(*')))., (120) 
^ {f{c^W){R-^vf^)^, (121) 

n**^-^(l-(sgn(/(<^(*)))).). (122) 

We make the following assumptions to evaluate the station- 
ary estimates. 

Assumption 1: (Time-translation invariance: TTI) The dy- 
namics reaches a time-translation invariant estimate: 



lim m*-*-* — m, 

lim C(*+^'^) ^ C^^\ 

lim g(*+^'^) = G(^). 



(123) 
(124) 
(125) 



If this property holds, the dynamics reaches stationary esti- 
mates. □ 
Assumption 2: (Finite integrated response: FIR) The inte- 
grated response 



(126) 



is a finite non-negative number, i.e., x < oo- D 
The integrated response is also called as a susceptibility. 
Assumption 3: (Weak long-term memory: WLTM) 

lim G^*'*'^ = (127) 

for any finite 121, lHO]. □ 
Since the response function represents the memory which 
means what happened to the system, the weakness of the long- 
term memory implies that the system responds to its past in 
an averaged way. The details of what takes place during finite 
stages tend to be washed away. 

We assume that the stationary estimate is unique and 
Assumptions [U - [3] hold. Note that violation of WLTM 
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immediately implies x — co. Under Assumption [T] the 
matrices C = (C'^^'^ and G — (G''*''* ') can be regarded 
as symmetric Toeplitz matrices, since we consider that the 
size of square matrices C, G, and D are sufficiently large, 
viz., t + 2 is sufficiently large. The matrix D = {D'-^'^ ') 
whose element is given by ( [89] l is also regarded as a symmetric 
Toeplitz matrix from 



s'.s) 



to''* •* ~ TO^ 

^ and 



!('*)+ C( 

limt^oo = ctq + - 2m + C(t)] = L>(^), where 

we again use Assumption [T| Therefore, we hereafter make the 
following assumption. 

Assumption 4: The matrices {C,G,D} and their powers 
are Toeplitz matirices. All of them commute each other □ 
In this subsection, we derive the following proposition under 
some assumptions. 

Proposition 5: Let Assumptions [T| - 14] hold, then soft-PIC 
converges to an unique stationary estimate whose BER is given 
by 

n = ^(^l-^Dzsgn(5(z))j, (128) 

where g{z) is to be determined by equations that describe the 
stationary estimate: 



(129) 

(130) 

(131) 
(132) 
(133) 

(134) 

(135) 

□ 

This result is identical to that from the statistical-mechanical 
analysis |45|, which is called the naive mean-field theory 14). 
If we put f{x) = tanh(a;/cr2), = and G = 0, the BER of 
Proposition |5] can then be rewritten as 



Bzg{z), 

c= T)zg{zf, 
Jm 



X 



Dzzg{z), 



5(z) = /(E + z\/F + Gg(z). 

F = 

G = 



1 + I3x 

erg + /3(1 - 2to + c) 



Px 



i + /3x 



Pb^Q 



with 



TO 



j Dztanh(E + zyF 



Dztanh^(E + zV F 
1 



a2+/?(l-c)' 
al + - 2to + c) 
[a2 + /3(l-c)P ' 



(136) 

(137) 
(138) 
(139) 
(140) 



where the identity. 



X 



(141) 



holds in this parameter settings. This result recovers that of 
the marginal-posterior-mode detectors [46 1. It may mean that 
the relationship G = 0, which corresponds to the case where 
the self-coupling term is vanished, is required to achieve the 
individually optimal performance. 



B. Derivation of Proposition^ 

Since the covariance matrix R of ([86]l becomes 



R = [Y.{~f3GT\D[y^{-PG 



(142) 
(143) 



by using Assumption H] we then have i?^ = R. Similarly, the 
reaction matrix T is represented as 



r ^{\ + fiGY^^G 

oo 

= -Y.^'PTG^. 



(144) 
(145) 



Since G is a lower triangular Toeplitz matrix, the T is also 
a lower triangular Toeplitz matrix, i.e., F^*''* ^ = F^*"*^ ^ for 
s> s' and F^"'^') = for s < s' . 

Under Assumption |3] it can be considered that the system 
responds to its past in an averaged way. We therefore consider 
the average of 0^*^ instead of limf_i.oo (Z)'*^^-*, which gives 



1 * 

— F 



/,(*' + !) 



t + 2 



--k + v + e + 



(146) 



t + 2 



J2 E r(*'^'^)/(0(^)), (147) 



t' = -l s=-l 



where we put k ^ 7^ EL-i " - tT2 Et'=-i 
and = tV2^l'=-i^^^'^ ■ Under Assumptions [T] and |2] 
EtL-i(G(*'*'')" is written by 



lim E (G^''*'^)" (148) 
t'=-i 

(149) 



t'=-i ^ti=-i t„_i=-i 

•••X C(*"-2.*>i-i)g(*i-1-*') 
t t 



=X lim E • • • E G(*'*^)G(*'*^) 



(150) 
(151) 
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Therefore, the last term on the right-hand side of (1147) in the 
t —i' oo limit is evaluated as 



(152) 



t-^oo t+2 ^ J\V) 
t' = -ls=-l 

OO , t \ 

=/E(-/^)"(,l™ E {G^'^'^T] (153) 

n=l ^ '' 



1 + 



/, 



where 



(154) 



(155) 



t'=-i 



Taking the t — > oo limit and using ( 11541 ). the average single- 
user process of (1147b becomes 



<P = k + v + e + 



1 + /3X 



/• 



(156) 



First, we evaluate fc, which corresponds to the signal part. 
If series k'^'']_\ A;*^°\ • • • converges, then k — k^°°\ Since k is 
obtained as fc = 1 - E^=-i G^*"* = 1 " l^xk we then 

have 

I = = E. (157) 

1 + Px 

Next, we consider the variance of Gaussian random variable 
V. The average of v is obviously = limt_j.oo J2\'=~i 
— 0. Using the definition of the persistent correlation 
c = limf_j.oo C{t), the variance of v is given by 



1 ^ ^ 
= lim 7 —J > > 

^ ' r^oo (r + 2)2 ^ ^ 



t=-l t'=-l 



((l + /3G^)-il?(l + ^G)-i)(*'*') (158) 

OO OO 

= E E + /3G^)"')^''((l + 13G^)-')^''^ 



s=0 s'=0 

X lim 



(t + 2)2 ^ 



+ - 2m + c) 
(1 + /3X)^ 



(159) 

(160) 
(161) 



where we here write ((1 + /3G^)~i)(*+^'*) as ((1 + 
l3G'^y^Y'^\ since (1+/3G^)"^ is a Toeplitz matrix. Namely, 
V obeys a Gaussian distribution with mean zero and variance 
F: t; ^ M{0, F). The Gaussian random variable v can be 
represented as f; = ^/Fz by using standard Gaussian random 
vai-iable z ^ 7V(0, 1). 

Limit as OO converges by using Assumptions [T] 
|2]and[3] We therefore have ip — limt_i.oo J2\'=^i 4>^* — 

^(-) and / = lim4^o,_j^EL-i/(<^^*'^) = fi^^°^^)- 
These relationships give / = /((/>). Applying function / to 
both sides of ( 11561 ) and letting g{z) = /, we obtain 



7(z) = /(E + zVF + 0"+G5(z)), 



(162) 



where G = /3x/(l + Px)- Th^ persistent overlap m = 
limt_j.oo Ti'*), which is identical to that of (11231 ), then becomes 



m = lim 

t—>-oo 

= (/), = / Dz.9(z). 



We then have the persistent correlation c: 



c= lim lim 

T—>00 t—>00 



?2\ 



Dzg(z) 



(163) 
(164) 

(165) 
(166) 



in a similar way. Since we assume Assumption |2] the inte- 
grated response x of (1126b is a finite non-negative number 
and is then given by 



X^lim V 

t'=-i 



de Tf 



(167) 



Bzzgiz). (168) 



For arbitrary function F(a;) that can be expanded around x = 
0, the bit error rate Pi, = limt^oo ^f|*^ ^'^^ evaluated as 

lim = - / DzF(5(z)). (169) 



Therefore, the bit error rate Pi, can be evaluated as ( 1128b . 

Setting 0*^*^ = 0, i.e., ^ = 0, we have then arrived at 
Proposition |5] 

C. Stability Analysis for Stationary Estimates 

If Assumption [3] (WLTM) is violated, the dynamics do 
not achieve the stationary estimate. For example, oscillatory 
behavior in the dynamics corresponds to this situation. In this 
subsection, we investigate where Assumption|3]is violated and 
obtain the following proposition. 

Proposition 6: The condition where WLTM is violated is 
given by 



13 



Bz 



dgjz) 
de 



> 1, 



(170) 



where g{z) and all parameters are identical to the set of (1129) 
- ([I35]). □ 
If we put f{x) = tsinh{x/a^) and — Q and G = 0, condition 
( 1170b can be simplified as 



/3 



[(72 + - c)]2 



Dz[l-tanh2(E- 



>1, (171) 



where E, F, and c are the same as ( |139b , ( 1140b . and ( 1141b . re- 
spectively. Equation ( 1171b coincides with the AT instability iJl 
of the marginal -posterior-mode detector P6l . This condition 
(|171b is also identical to the microscopic instability, which is 
derived by the belief update of the BP-based detector [22|. 

In the rest of this subsection, we derive Proposition |6] Let 
us separate the response function G*-* * ' and the Onsager 
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reaction term F^* * ^ explicitly into TTI parts and further small 
contributions: 

Q(t,t') ^Qit-t') ^^{t,t')^ (172) 
p(t,t') __p(t-t') (173) 

with limt^oo^G(*+^'*) = and limt_>oo r(*+^'*) = f(^), 

where G'*"*' and f(*~*') denote TTI parts that are re- 
ferred to as persistent parts. The G^*'* and F^*'* ^ denote 
small contribution terms that are referred to as non-persistent 
parts. If weak long-term memory holds, then G'^* * ^ and 
p(t,* ) vanish for /; — > oo. We here assume that some 
small long-term memory, but such that the limits G*^* ^ = 
limt_,oo and f(*') = limt„,oo f exist and also both 

X = limt^oo E*'=-i G{t'), 7 = limt^oo H^')^ X - 

limt^oo G'(i') and 7 = limj^oo f (i') exist. 

When Assumption |3] holds, the identities limf^oo G^*'* = 
and limt^oo f = hold. 

We expand (|87] l for small G, and find 



n— 1 'm— 



^71— m 



(174) 



(l+/3G)-^/3G 

(-^G)'"(^G)(-/3G)"-i-™ + 0(G2) 



00 n— 1 



n— 1 m— 



(175) 



where the {t, t') elements of the (t+2) x (t+2) matrices G, G, 
f , and f are G'*^*'', G**'*'), f (*>*'), and f respectively. 
From (|175t . the persistent and the non-parsistent parts of T is 
given by r = (1 + (3G)^^I3G and F. Then, 7 becomes 

t i 00 

7=/3 1im ^ ^ ^(n + l)(-/3G(*'^))"G(^^*') (176) 

t' = -l s=-l n=0 

/3X 



We next substitute ( |172| i into (11 18t and take the average with 
respect to iterative steps t, i.e., X]*'=-i' both-sides of 
the substituted equation, which gives 



I3x 



1 



1 + /3X t 



(178) 

As limt_i.oo G^*'* \ since we assume Assumption |2] (finite 
integrated response), we find 



G(*) = lim G(*'*') 

^limM^ 

= liin/f^V 

t^ooWa^y f + 2 



1 * 
— y f 



s=-l 



(179) 
(180) 

(181) 



Substituting this equation into the definition of Xj the follow- 
ing identity is found: 



Although X — J — always solves this equation, one finds 
another solution when {j / x) {{d f / 09)'^) ^ — 1 holds, which 
is equivalent to the AT line [l]. Since x = is required to 
hold Assumption |2] and ( I182l i represents an evaluation of x, 
the evaluated value should not be greater than x^ i e., x < 
l{{df I d6Y)v, to diverge stationary estimates. We therefore 
have an instability condition of Assumption |2] as 



dl 
86 



^ >1 



(183) 



by substituting ( 1177b . Using X = X + X 
have then arrived at Proposition |6] 



X and (1132b . we 



IX. Decoupling Principle 

With the statistical-mechanical approach |46l, (TT\, fTSl, 
a complex many-user problem such as an inference problem 
where many users' transmitted signals in the CDMA system is 
reduced to an equivalent single-user problem. This approach 
brings a significant interpretation in the communication theory. 
This is known as the decoupling principle found by Guo and 
Verdii [TT|. It claims that the vector channel concatenated with 
optimal detection is equivalent to a bank of independent single- 
user additive white Gaussian noise channels, whose signal-to- 
noise ratio degrades due to the multiple-access interference, in 
the large-system limit under a certain randomness assumption 
of the channel. 

The decoupling principle has recently attracted a great deal 
of attention and has been investigated in detail 1211 . ||37]| . 
Especially, by applying density evolution, Ikehara and Tanaka 
have found that the decoupling principle holds not only at 
equilibrium but also at each stage of the BP-based detector 
II2TII . Their analysis was however based on an assumption 
of independence of messages. The GFA allows us to study 
algorithms without assuming the independence of messages 
E, Q, ||27l, ED, ill. In this section, we investigate the 
decoupling principle via detection dynamics using GFA by 
considering an arbitrary fading model. 

A. Generic Detection Algorithm 

In this section, we focus on a generic system model and a 
generic iterative multiuser detection algorithm. 

We now consider a iC-input TV-output vector channel, which 
can be regarded as a fully-synchronous X-user CDMA system 
with spreading factor or a multiple-input multiple-output 
(MIMO) system. Let 6 = ,6^)^ e denote the 

input vector, and y = {yi, ■ ■ ■ , un)^ € denote the output 
vector of the channel. The output y is given by a linear 
transform as 

y = Sb + n, (184) 

where S = (aiSi,--- ^ukSk) denotes w N y. K chan- 
nel state matrix, which includes the received amplitude a^. 
Sk = -^{sik, • • • , SNk)^ denote the channel parameters for 
the spreading code sequence of user k, and n is a vector 
consisting of i.i.d. zero-mean Gaussian random variables with 
variance Cg. We use control parameter ct^ as the estimate 
of the channel noise variance instead of true noise variance. 
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Each element s^^ G { — 1,1} is an i.i.d. random variable 
with equal probability. Also, let pb{b) = Y[^=iPb{bk) denote 
the joint prior distribution of input b S {—1, 1}^- Input 
distribution Pb{b) is arbitrary. The fading model is represented 
by arbitrary probability distribution Pa{o.) = IlfcLi Pa(flfe)- 
The distribution Pa{cL) is assumed to have finite moments. 
Especially it has the second-order moment 7. We analyze the 
system in the large-system limit where K,N ^ 00, while the 
ratio /3 — K/N is kept finite. 

Definition 5: (Generic iterative algorithm) The updating 
rule for tentative decision 6^*-' e M of bit signal bk at stage t 
is 

^^'^ = (185) 



function: 

/(a;) =tanh(a;), (191) 

={c72+/3[l-g(*)]}-\ (192) 

+ A(*)^[1 - Q(*)]i?(*-i), (193) 

ei'^ =0, (194) 

ri'^'"^ ^~Ss,s'A^'\ (195) 

t 

J^' =l{0 < s < t)A^'^ Y[ A^''">l3[l-q(''^] 

s'=s+l 

+ I{s = t)A^*\ (196) 



with 

s=-l k=l 

-i:^t'b':'+0i'\ (186) 

s=-l 

where h = {hi, ■■■ , HkV , W = {Wkk'), R^*l A^'l 
r^,''^-*, and 6'(*'' are parameters. Transfer function / : M — > M 
is arbitrary and applied componentwise. □ 

The is a summation of messages from tentative de- 
cisions. The 6*^*^ is again introduced to define a response 
function. Let hk be an output of the matched filter for user k, 
i.e., hk — Sk ■ y, and let W he a K x K correlation matrix 
which is defined by W = S with S = (si,--- ,sk)- 
For initialization, we use b^. — 0. This generic algorithm 
includes various types of iterative multiuser detectors as in the 
following examples. 

Example 2: If we set the transfer function / and the pa- 
rameters as 



with Q(*) = K-^T,k=i[bk''?- Here, l{r) denotes an indica- 
tor function, which takes value 1 if P is true, or otherwise. 
We assume that Q{t) can be regarded as a constant with 
respect to b by using the central limit theorem. Then (I185l l 
is equivalent to the BP-based detector ll22l . □ 

As previously mentioned, the iterative detection algorithm 
(I185l l includes various types of iterative algorithms. It should 
be noted that in all these detectors, gives an approximate 
value for the posterior- mean at the t^^ stage ||471 . ||221 . We 
obtain the analytical result, which involves the following 
measure; 



xS[bir'^] n S[bi^^'^-f{ui^^)], (197) 

s=-l 



with 



f{x)=tanh{x), (187) 
=^7"^ (188) 
jis.s') =^-^Ss,s', (189) 

A(^) =ei'^ = ti'^''^ = 0, (190) 

the algorithm of (fT85] l is identical to soft-PIC JH): = 

tanh[^(/ife - Xl^=i W^fcfc'^i*^)]- Here, 6s,s' denotes Kro- 
necker's delta taking 1 if s = s', or otherwise. □ 

Example 3: All parameters except Tj^ ' ' use those of soft- 

" (s s' ) 

PIC. If the ' ^ are chosen to cancel the Onsager reaction, 
then algorithm (fTSST i gives ORC-PIC gT). □ 

Example 4: We next consider the following parameters and 



ui'^^akU%k + vi'^+ E l^^-'-'^-^^k'^'W^ (198) 



where g{bk,Vk) denotes an arbitrary function of bk = 
ib'f^\--- ,bl^^y and Vk = The parame- 

ters are as follows: 



R={1-PB)-'^D{1-I3B'^)-^, (199) 

t-i 

T = -[J2 -Js+i<» A^+i((7^)-y (200) 

s=-l 

fc^^^ = |A,|, (201) 
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where B, U, and D are matrices with elements 

j{s.r)Q{r,s')^ (202) 

T = -l 

S 

U^"^"') =5s^s'+P j('^^)g^^-''^ (203) 

r' = -l 
s s' 

+ Y J("'^)J("''^')C(^'^')], (204) 



^ t' = -1 

+ I(s' = s)i?('''). 



(205) 



with Pi) = \27rR\^^/'^dv exp[—^v ■ R^^v]. All these param- 
eters mentioned above are obtained from the following three 
kinds of quantities: 



m 



is) 



1 ^ 



k l*k 



k=l 



K^oo K 

K-fOO K ^ " " 
k=l 

~\"k "k /*k,bk,ak^ 



(206) 



(207) 



fe=l 

=I(s<s')(6i^)(H-it;fe)(^'))*„6„a., (208) 

where G''^'" for s < s' due to causality. Here, A = 
{Ss+i,s') is a (i + 1) X (i + 1) matrix. Operator ( • )u denotes 
the expectation with respect to random variable U and operator 
(g5 denotes the Hadamard product, i.e., A (E) B — {uijbij) 
for A = (ttij) and B = {bij). Terms {R^^vY^'> denote the 
s**^ element of the vector R~^v. Term X]s'=-i T^^'* ^ in 
( |198t represents the retarded self-interaction, which is called 
the Onsager reaction. 



B. Decoupling Principle 

From the average of (1197) we find the effective single-user 
process: ^j^"^^'' = f{u\^^) and b[. = 0. Variable can be 
regarded as the {t + l)-dimensional Gaussian variable with 
mean vector and covariance matrix R. Using ( 1198b . the 
effective single-user iterative process can be written in the 



bi ■ 

b2- 





Vector channel 


y 


Multiuser detector 



















-b1(t) 



(a) 



bi- 
b2- 

bK- 



Scalar channel 



Scalar channel 



Scalar channel 



4 Single-user detector 



\ Single-user detector 



\ Single-user detector 



— b2(t) 
-bK(t) 

-bi(t) 
• b2(t) 

-bK(t) 



(b) 



Fig. 4. Schematic of decoupling principle, (a) Vector channel followed by 
multiuser detection and estimates at stage t. (b) Equivalent scalar channels 
followed by single-user detectors and estimates at stage t. 





Vector 
channel 


y 


Multiuser 
detector 











-b(t) 



(a) 



bk(-1) 



bk(t-1) 



Effective ^ 

, Gaussian 

='9"^' noise 
amplitude 

akk(t-1) N(0,R(t-1,t-1)) 



— ^ — 







-rk(t-i,-i) Y 

Onsager ^ 
reaction 


; 


I r(t-i,,-i) i 
f -rk(t-i,t-i) 







Single-user 
detector 



■ bk(t) 



(b) 



^""'^''f Gaussian 
noise 

amplitude 

akk(t-1) N(0,R(t-1,t-1)) 



bk- 











Single-user 
detector 











■ bk(t) 



(c) 



Fig. 5. Vector channnel and equivalent single-user channels, (a) Vector 
channel, multiuser detection, and tentative decision, (b) Equivalent single- 
user non-Gaussian channel, single-user detection, and tentative decision, (c) 
Equivalent single-user Gaussian channel, single-user detection, and tentative 
decision. 



following simple form. 

4'^ ^akk^''>bk + vi''> 

E [r'^^^'^-r^^'V(4''"'^), (209) 



s' = -l 



with ^ Af{0,R'-'-'^-'-'^'^), where 6^"^ /("i""^^), 
Uf, ^•^ = f{u[. ^'') = 0. Introducing the average over 
Vk as {■)vk ~ jT^Vk- and using ( |209t . then m^*', 
C^"'"'), and G^''-"') are obtained as m(") = (6/cfoi'^)«,6,,a,, 
Cis.s') ^ {bi%i''^)^M,^a„ and G(^^^') = I(s < 
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s')(b'"j^\R~^Vk)''''^ '^)vk,bk,at^ respectively. The macroscopic 
parameters contain the effects of all users as averages; there- 
fore, these can be represented by each user's distribution. 
Hence, ( |209l l holds for any user k. All the parameters in (1209) 
can only be represented by each user's distribution. 

This result (|209l l means that each user experiences an equiv- 
alent single-user additive noise channel whose signal-to-noise 
ratio degrades due to both the multiple-access interference and 
the Onsager reaction, at each stage of the algorithm. This 
property is called the decoupling principle IHTI . The additive 
channel noise generally becomes non-Gaussian due to the 
existence of the Onsager reaction X)t'=-i T'*^^'* "'/("i* ^'')- 
Figure |4] has a schematic of the decoupling principle. 

Figure |5] shows the results of the analysis. Approximate 
posterior-mean estimator (PME) ^j.*' = ^'') is used as 

the single-user detector It should be noted that (1209) implies 
that equivalent channel noise becomes Gaussian if the Onsager 
reaction is cancelled. However, for soft-PIC, the Onsager 
reaction is not cancelled (Fig. |5] (b)). If the Onsager reaction 
does not vanish, the noise distribution is not centered at zero. 
Therefore, soft-PIC must be suboptimal. 

Under the assumption of the large-system limit, the perfor- 
mance of algorithm ( I185l l at stage t can be evaluated by using 
the equivalent scalar channel determined from three types of 
macroscopic parameters (m, C, G), where m, C, and G are 
a (t+l)-dimensional vector {m^^^\ ■ ■ ■ ,m(*^^^)^, a (t+l^x 
{t+1) matrix (C'"^"')) and a (< + l)x (t+1) matrix {G^"'" )V 
respectively. The bit error rate P^'*^ of hard decisions sgn ] 
at the stage is given by F^**^ = 5(1- (6fc sgn [&i*^])^,b^,a J- 

We next consider the case where we set the parameters 
in (1185b to those of ORC-PIC. The reaction term in the 
effective single-user process is determined by two matrices 
r and Ffc. It is considered that the parameter F represents a 
retarded self-interaction caused by iterative calculation. The 
parameter Ffe, on the other hand, is arbitrary; therefore we 
can choose Tk that cancel the Onsager reaction. Using the 
inductive method, we can show that the Onsager reaction 
is entirely cancelled at each stage t, when the parameter 
f is set to (fb)(*) = ;3G(*'*-i)[S(*-i) - (fb)(*-i)] and 
(f 6)("i) = (f = 0. This parameter f coincides with the 
parameter derived by density evolution |47 |. Density evolution 
cannot treat the Onsager reaction, however density evolution 
can be applied to derive an algorithm that can correctly cancel 
the Onsager reaction. 

In the case where we set the parameters in ( 1185) to 
those of the BP-based detector, we confirmed the Onsager 
reaction is cancelled at least at stage 8 through straightforward 
calculations. In this case, F becomes diag(^(^^\ 0, • • • ,0). 
Therefore, for both ORC-PIC and the BP-based detector (at 
least at stage 8), the equivalent single-user channels are given 
as an additive white Gaussian noise channel (Fig. |5](c)). 

X. Conclusion 

We evaluated the detection dynamics of soft-PIC, ORC- 
PIC and the BP-based detector by applying GFA in the 
large-system limit. We also showed that GFA could treat the 
dynamics of beUef propagation. 



From the practical point of view, iterative algorithms are 
generally utilized as suboptimal methods. It is important to 
understand the detection dynamics in detail to improve detec- 
tors. We confirmed that the results from the density evolution 
analysis could be obtained from those of the GFA analysis by 
omitting the Onsager reaction term. The GFA could correctly 
treat the Onsager reaction, which is a retarded self-interaction 
due to iterations, and gives density evolution a basic theory. 

We also studied the decoupling principle in iterative mul- 
tiuser detection algorithms using GFA. and found that the 
decoupling principle holds. The reconstruction algorithm of 
compressed sensing could also be analyzed by GFA L32J . 

Appendix A 
Notations 

We here summarize notations, which are used in this paper, 
in Table U It should be noted that notations to describe the 
GFA result don't contain the user number k, since a many- 
user problem is reduced to an equivalent single-user problem 
in GFA. In Table U 'BP' denotes BP-based detector. 
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Appendix B 
Derivation of Lemma[T] 

Evaluating the averaged generating functional straightfor- 
wardly, one can obtain Lemma [T] Substituting ^ and Q into 
in} , the summation over all messages becomes 



,(t) 



K 

E 

k' = l 



lit) 



N 

M=i 



(210) 



table I 
Summary of notations 





the first 




variable 


or top-left 
element 


description 


(notations for models) 






K eN+ 




the number of users 


N e'M+ 




spreading code length 


k,k' e {!,■■■ ,K} 


- 


user number 


e {!,■■■ ,N} 


- 


chip interval index 


t G N+ 


- 


stage 


s,s',t' G {-1,0,--- ,t} 


- 


stage 


Sk = «) G {±1}^ 


*fe 


spreading code of user k 


b = (bk) G {±1}^ 


bi 


information bits 


foW = (fcW) GR^ 




tentative soft decisions 






tentative hard decisions 


y = (j/M) G 




received signals 


n = (n'') G R^ 




noise vector 


UQ G M+ 




true noise variance 


CT G IR+ 




control parameter 


W = (H^fcfe/) G R-f^x-ff 


Wii 


correlation matrix 


V>(*) = (V^*') G R^ 


V'i 


dummy functions 


= (e^) GR^ 




external message 


h = (hk) G R-^ 


hi 


matched filter outputs 


/ : R -5- R 




transfer function 


ffc = (fj^"'"'>) GR(*+iW*+i) 


p{-i,-i) 


parameter for ORC-PIC 


G R+ 




parameter for BP 


G R+ 




parameter for BP 


G R+ 
j{t,s) g 




parameter for BP 




parameter for BP 


7 G R+ 




inverse temperature 


S=(afeSfe)GR^x-^ 


aisl 


channel state matrix 


aj. G R+ 




received amplitude 


Pj^*' G R+ 




bit error rate 



(notations for GFA) 

6 = (6(0) eRt+i 
S = (fe(*)) G {±i}*+i 

tP = (^(t)) 6 Rt+1 

9 = (6lW) G R*+i 

ri = (rj^")) G R*+i 

fi = (jjf^)) G R*+i 

fc = (feC")) G R*+i 

fc = (fe(«)) G R*+i 

q = G R(*+1)x(*+1) 

q = (g(s.s')) G R('+l)x(t+l) 

Q = (Q('''''')) g ]R(t+l)x(t+l) 

Q = {Q(-='='^) G R('+l)x(t+l) 

L = (LC"'"')) G R(t+l)x(t+l) 

L = (Lf'''''')) G K(*+iMt+i) 

m = (m(=)) G M'+i 

C = (Ct"'"')) G K(t+i)x(t+i) 

G = (G^"'"")) G m(*+i)x('+i) 



&{-!) 
&{-!) 

^(-1) 
^(-1) 

Q(-i.-i) 

c(-i.-i) 

G(-i.-i) 



tentative soft decisions 
tentative hard decisions 
dummy functions 
external message 
macroscopic parameter 
macroscopic parameter 
macroscopic parameter 
macroscopic parameter 
macroscopic parameter 
macroscopic pai'ameter 
macroscopic parameter 
macroscopic parameter 
macroscopic parameter 
macroscopic parameter 
overlap 

correlation function 
response function 



The average generating functional Z[if)] is represented using 
it'^*' = {u'"i\--- jU^'')^ whose elements are defined by 
illOi . Using Dirac's delta function S, we first introduce the 
definition of it'^^'. • • • ,tt(*^^) into Z[ijj], which is written in 
terms of them, as 



du 



(t + l)K 
^s— — 1 ^ 



X exp 



t-1 K 
k=l 



(s) 
k 



K 



N 



k' = l 



^k 



(211) 



where dM=n,=_i Jlfe^i 



The arguments of Dirac's 



delta functions (5( ) represent the definition of {m^ }■ The 
term of 5 is rewritten by applying the Fourier integral form of 
Dirac's delta function 5{x) — /jgdie'^^ as 5{\ 



Ir '^^i*^ 6'^P{i'"fe'''('"fc^'' — [• • • ])}• We then have 



b(-i),--- ,b(*) 

X exp 

t K 



s=-l k=l 
t K 



s=-l k=l 



7 _ 7 rr(t+i) 



2n 



M=l s=— 1 ^ ^ fc=l 
W t-1 X AT s 

-'"EE ;^E<4-' 

^=ls=-l ^ V J^V j,^;^ / 



(212) 



where (116=]^* Ilfe^i ~^2^- Without loss of generality, we 

can set to bk ~ 1 (Vfc). 

To take the average of Si, • • • , sk, we introduce variables 
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t;}f'' and w^*-' which are defined as: 



Since t is finite, term Eg^,... ,sjc{- • • } in (|215l l is given by 

I N i-1 K 

Esj,... ^s^l cxp 



(s) A 



(213) 
(214) 



^=1 s=-l k=l 



N K 
II II COS 

M=l /c=l 
N K 



It should be noted that the averaged generating functional Z [ip] 
of ( I212I 1 includes Si, • • • , sk only in terms of these variables 



/i = l A;=l 



t-1 



K 



is) (s) 

vli and wit . Due to this, random variables si 



1 1 



st^ are 



(s) (s) 

isolated and are able to be averaged. Introducing u/j and w/, 
into term E 



AT 



t-1 t-1 



s 1 , ■ ■ ■ , s K , n 



{• • • } in ( 1212b . one obtains 



p=i 



s=-l s' = -l 



if 



fc=l 



/ 



dfdiDE, 



/R2tJV 

exp 



s 1 , ■ ■ ■ , s K , n 

TV i-1 



AT t-1 

iV^-oEE-l^^""-i/^EE-l^' 

/i— 1 s— — 1 /J.— 1 s— — 1 

k—1 
K 



^ k=l k=l ' 



n - (s) 



k=l k=l 



A' ^ K 

k=l k=l 



xSl W 



;^E<{i"^i"} 



fc=l 

dvdvdwdw 

r Af t-1 

X exp i^ ^ {«(^)«(^) + - /3z;(^)«;(^)} 

/I— 1 s— — 1 
, p Af t-1 

xE„< exp 

L ^=is=-i 

AT t-1 



fe=l 



(217) 



We next separate the relevant one-stage and two-stage values 
which are appeared in (1217) : 



is) 



i^ao EE"; 

fl—1 S — —1 

N 

" s= — 1 fc=l 



^E^i' 
^^^^ = ^E4' 



fe=i 



xE 



'^-7^ Err, 



AT 



fc=i 



(215) 



fc=i 
1 ^ 

A 1 y 



fe=l 



k "fc ■ 



(218) 
(219) 
(220) 
(221) 
(222) 



Equation (1217b is only written in terms of these variables 
We here again used the Fourier integral form of Dirac's delta, ^(s) j^{s) q{s,s') q(s,s') ^j^^ /^(s,^') hereafter call 

The term E„{ - • • } in ([21511 becomes these variables macroscopic parameters, which describe 

the nature of the system and are also often called the 
order parameters. Similar to the way to derive (1215b . 
one can introduce m''^\ k^''\ g^"'^ \ Q*-*'^ and i'**'" ' 
into (1217b . The term of S in (1217b is again rewritten by 
applying the Fourier integral form of Dirac's delta such 
as 5(?7('*) -[■■•]) = /r dryf") exp{i?7(")(7/(^) -[■••])}, 
TTexpLVo^F y vl^^vif^]. (216) ^ki^^ -[•■■]) = L exp{ifc(^)(fc(^) -[•.•])}, 

li ^ ^ ^ -[...]) = 4 dq(^>^') exp{ig(-^')(g(-^') -[•••])}, 



E„<^ exp 



AT 



N t-1 

-iV^^oE E "^''^'' 

/I — 1 s — — 1 

t-1 f-1 



Ai=l 



s=-l s' = -l 
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S{Q(^'-') -[...]) dg(^'^') exp{iQ(^'^')(Q(^'^') 

-[...])} ^and 6{L^^^^'^ -[•••]) = 4 dL^^'^'^ 
expjii'^*''' ) — [• • •])}. We here introduce the notations 

^ nli-i ^ = uT=\ d^ ^ n:=-i d^(^^ 

d»7 = nl=-i d??(^^ dk ^ YlTJ-i dfc(^), dfc ^ n*=-i dfc*^^ 
dg ^ Utl^ Ul^U dg(-'), dg A Yll-J_^ nri_i d<z(-'), 

dQ ^ n*=-in*7i_idg(^^^'), dQ ^ 
n:=-in:7i-ido(^'^'\ dL ^ n:;-in 

t-l T-rt-l iffs.sM J.. A r-iN 



dL ^ 
dp ^ ^ 

d- ^ U:.^UlrJ^^ 



^^_idL(^^"') 

1 

-1 



dv 



uu n 



2ir ' — 



dto = 



n;Lin:: 



l^i-idi^"'"'' 
-1 " 

1 dm^°' 
-1 



/27r 



/27r ' 

and 



-1)1 A 



Since the initial probability, 

which is given as = HfeLi ^(^i^^'')' factorizable, 

the averaged generating functional Z[ip] factorizes into 
single-user contributions, i.e., with respect to user index k. 
We have then arrived at Lemma [T] 

Appendix C 
Derivation of Proposition[3] 

The inductive method is applied. 

(i) The case of f = — 1. This is an initial stage, therefore the 
Onsager reaction term does not exist. To cancel the Onsager 
reaction term, the matrix Tb is simply chosen as (TbY^^^ ~ 0. 
This is automatically held by the initializaiton M^^^ = 0. The 
matrix G is G = because of causality. 

(ii) The case of t = 0. In this case, G is a 1 x 1 zero 
matrix O and T becomes T (1 + /3G)^i/3G = O. The 
Onsager reaction term is therefore Tb = O. To cancel the 
Onsager reaction term, we only have to choose matrix T as 
(f6)(-i) = 0. Mati-ix G is 



G== 







(223) 



until here. 

(iii) The case of stage t ~ 1. Since G £ K^^^ is a nilpotent 
matrix, i.e., G^ = O, then the Onsager reaction term Tb 
becomes 



Tb = [{1 + (3G)-^ l3G]b 
= pGb 



/3G("^-i) 








6(0) 



(224) 



To cancel the Onsager reaction term, one needs (fb)*^"-' — 
0. If {tb)^°'> is chosen as this, the average ((fo*^^^)) ^ J Vv 
sgn +9^^''] only includes O^^'' and does not depend 

on 6l(-i). Therefore, G^^'-^'> = d{{U^'>)) /d9^-^'> = holds. 



The matrix G is 









G= I G(0'-i' 
G(i'") 



(225) 



(iv) The case of stage t — 2. Since G G R'^^^ is a nilpotent 
matrix, i.e., G^ = O, then the ORCT Tb is 



r6 = [{1 + I3G)-^ l3G]b 
= [/3G-(/3G)2]6 



/3G("'-i) 

„^2^(l,0)g(0,-l) ^^(1,0) 




(3G'^h0)l{0) 

Therefore, if Tb is chosen as 

(f6)(i) =/3G(1'")6("), 





6(0) 
6(1) 



(226) 



(227) 



then it cancels the Onsager reaction term. When we choose 
(fb)(i' as this, the average ^ JVv sgn [k^^'> +v^^'> + 

6''-^'] only includes 6(1) and does not depend on 0^^^) and 
6'(0). Therefore, G^^.'*) = d{{U^'))) /d0^-'^'> = holds for s e 
{—1,0}. The matrix G is 



G = 



/ \ 

G'-^^-^^ 

G(i^") 

\ G(2^i) / 



(228) 



until here. 

(v) The case of stage t — 3. Since G G M'*^'* is a nilpotent 
matrix, i.e., G"* = O, then the ORCT Tb is 

rb = [(1 + /3G)-VG]6 

= [/3G-(/3G)2 + (/3G)3]6 

/ \ 



(3G<-mU°^ 

\ /3G(2.i)6(i)-/32G(2.i)G(i'0)&W ; 



(229) 



Therefore, if f 6 is chosen as 

(f6)(2) = pG^^'%W _ ^2^(2,1)^(1,0)^(0)^ (230) 

then it cancels the Onsager reaction term. When we choose 
(f6)(2) as this, the average {U^^} = JVv sgn [fc^^) + 
^(2) _|_ ^(2)j Qjjjy includes 6(1) and does not depend on 
e(-^\--- ,0(1). Therefore, G^^^'^ = diU^^)) /dO^-^^ = 
holds for s G {—1, 0, 1}. The matrix G is 



G = 



/ \ 

G(0'-i' 

G'^^'°^ 

G(2'^' 

V G(3.2) / 



G(3.2 

until here. Note that it is expected that 

(f6)(*) = /3G(*'*-i)[6(*-i)-(f6)(*-i) 



(231) 



E(-^)*"in^*"^'"M^^'' (232) 



until here. 



will holds for any i > 1 in an analogy with ( I230l l. 
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(vi) Stage t > 3 

We here assume that the Theorem holds for stage 
t e { — 1, 0, 1, • • • , i}. Namely, we assume that 
(fb)(-i\ • • • , (f6)(*) are given by 

(f = 

(fb)(°) = 



argument of tanh(- • • ) of (fT4l i becomes 



K 



fc'=i 

t K 



s=-l k' = l 



+ e 



is) 



(237) 



(fb) 



(t) 



s=0 



t-1 



Substituting ([T]! and (O into ( 1211 1. the averaged generating 
functional Z[i()] is represented as 



and G has the following form: 



G = 



( 

G(o.-i) 

G(i,o) 



V 



\ 



G(*+i^*) / 



(234) 



Since G e M(*+3)x(*+3) is a nilpotent matrix, i.e., G*+^ = O, 
then Onsager reaction term Tb e M*+'' can be calculated as 

= [/3G- (/3G)2 + ... + (-l)*+i(/3G)*+2]6 

/ \ 



/3G(2,1)5(1) _ ^2^(2,1)^(1,0)^(0) 



xp[b(-i)]f n 

r * 
X exp -i ^ 6'^'*'' • i/j* 

t-1 if 



27r 



a: 



S=0 ^T = S ^ 



/t-1 



s=Ofc=l ^ k' = l 

t K N 

s=— 1 k' = l ^ u=l 



(238) 



(235) 



If rb is chosen as 



(fb)(*^ 



J|G("+i-)UW, (236) 



where dM=J|*^Q J^^j^ ~^2n' Wi'^hout loss of generality we 
can put bk = 1 (Vfc). We apply the Fourier integral form of 
Dirac's delta function, which gives 



b(0),... ,b(t) 

t-1 K 

X exp 

L s=o fc=i 
t X 



the Onsager reaction term is cancelled. When we choose 
(fb)(*+i) as this, the average ^ J Vv sgn + 

only includes 6'*^*+^-' and does not depend on 
6»(-i)^... ^0{t\ Therefore, G(*+2^'^) = a((6(3)))/06»(-i) = 
holds for s e {-1, 0, 1, • • • , t}. 

If the claim holds for stage s E { — 1, 0, 1, • • • ,t}, it holds 
for stage t + 1. This proves Proposition |3] 



Appendix D 
Derivation of Proposition^ 

One can evaluate the BP-based detector in the same manner 
as soft-PIC. The summation over all messages, which is the 



s=0 k=l 



X E 



Si,-- - ,sk GXp 
t-1 



^=1 s=0 ^^^^ k=l 



(239) 
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where dtt^^JI*^^-,^ JlfcLi ' ^'^^^ average of 

Si, • • • , sk, we introduce the following variables: 



Term ... ^sk{' ■ ■ } in ( I242l i becomes 

I I N t-l f K 



K 



(240) 



—y 

E (241) 

/c— 1 ^ s' — — 1 



It should be noted that the averaged generating functional Z [■0] 
of (l24l) only includes si, • • • , s/^ in terms of these variables 

(s) Is) 

vli and wli . Due to this, the random variables Si, • • • , sk 
are isolated and their average is able to be taken. Introducing 
v\^^ and ivj^^ into the term Eg^^... _sj^ „,{••• } in ( |24] |. one 
obtains 



dtldKjEsj^...^SK,'n. 



TV t-1 TV i-1 

iV^-oE E i?^^^-l^^---i/3E E -^^^-^^^^ 

/i— Is — — 1 /I— 1 s— — 1 



AT if 



;^EEk^E<4' 

k=l s' = -l ^ ■' 



nn-P 

^i=i fc=i 



1/1 



E 

s=0 



s' = -l 

r t-1 i-1 . / 1 ^ \ 

n-p A^T.ki^Y.<''-'^')M^') 

u=l L s=0 s'=0 ^ fc=l ^ 



(244) 



+ ■ 



1 ^ 

k=l 

K 



exp 



- E '^^^'^'4^E^i^'"H)-.(^') 

r' = -l L fc=l -"^ 



^ fc=i 



fe=l 



X 6\ w 



fc=l = 



X exp 



dvdvdwdw 

r N t-l 

fl—l s — — 1 

TV f-1 



r=-l L -1^ 

^ r'=-l 



K 



k=l 



T = ~l 



k=l 



xE„|exp -i^aoJ2 E ^^'^^^1'^"" | 



fe=l 



(245) 



X E., 



exp 



-ir^EEfe'E'^^'"'"^ 



M=l s=-l ^ fc=l 



k=l s' = -l ^ -I ^ 



(242) 



Where dv = 11^=1 Il.^-i 7=^ = 11^=1 Il.^-i 7=, 

= n^=in.=-i^ and dw = n^^in.^-i-^- 

We here again use the Fourier integral form of Dirac's delta. 
Term E„{- • • } in ( I242l i is given by 



for finite t. The relevant one-stage and two-stage values, which 
are equal to those of soft-PIC's analysis, appeared in (I245l l 
are separated by introducing (1218) - (I222K The term of S in 
(I217l i is again rewritten by applying the Fourier integral form 
of Dirac's delta function as well as the case of soft-PIC's 
analysis. 

Since the initial probability, which is given as p[b(^^^] = 
IIa^i '^(^fe ^')' is factorized, the averaged generating func- 
tional Z[^] factorizes into single-user contributions. The 
averaged generating functional is therefore simplified to 
Z[ip] = / dr/dJ7dfcdfcdqdqdQd(5d£diexp[ii:($ + * + l^) + 
0{\nK)], in which functions $, ^I*, 51 are given by 

t-i 



En< exp 



N t-1 
fj,=l s=0 



* =i E {fj^'^V^"'^ +k^"^k^"^ 



t~l t-1 



N 



U.= l 



s=0 s'=0 



iE E{^^^^' 



(243) 



s=~l s'=-l 



(246) 
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K 



k=l 



X exp 



t-i 



'-s=-l 
s' = -l 



s=-l 

t-1 



(247) 



the single-user measure 

Wk{{h,u,u}) 



^ 5[b^~^^] exp 



E{i- 



' zvr ^ 



s=0 s'=0 

s=0 
t-1 

-iE' 

saddle 



(249) 



= — In / dvdvdwdw 
K 



X exp 



N t-1 



j_L—l S — —1 

N t-1 t-1 



-^EE E ^-^^^^^^'"H^'-r 



^ r=-l ' 



T = -l t' = -1 



(248) 



instead of the single-use measure (IJTT i for soft-PIC. 

The integral in the averaged generating functional Z[ip] will 
be evaluated by the dominating saddle-point of the exponent 
$ + ^t* + in the large system limit — > oo. Using 
the identities ^ - ^ and gT]) - ^ with ( |249l ). the 
differentiation of <i> + 'J + 51 with respect to 77'^"^ t)*^*^ k'^'^\ 

kis)^ qi^.s')^ qis,s')^ Qis.s')^ Qi^.s')^ l(s.s')^ ^jjj l{s,s') jg^^^j^ 

US to the following saddle-point equations ( l47b - ( l56] l with 
( I249I 1. respectively. Comparing these saddle-point equations 
with (l44l l - ( l46l) . we again find the following relationships: 

= to("), g^'''"') ^ C^"^""), and L^''-"') = iG^"^""). 

The integral in with respect to i) and w is given by 
= -^{hi |D| + In |g + - B)T£)-i(/3-il - 

where matrices B and £) whose (s, s') elements are given by 



^(^,«') A_ _ ^ j(.,r)(^(r,.)^ (250) 

r = -l 

r' = -l 

T = -l t' = -1 



respectively. 

The saddle-point equations including il are evaluated as 
One can deduce the meaning of macroscopic parameters by foUows. One finds rj^-'^ = and g(^) = 0. We put 



differentiation of the averaged generating functional ^[■0] with 
respect to 6^f^^ and ■^i*''- The averaged generating 
Z[il)] is dominated by a saddle-point for K ^ 00 



(252) 



Applying the same way as the derivation of soft-PIC, we 
again obtain i4T[ - (03]), with the average dJTl i ( )k which has for light notations. Noting that {7 includes G via B and using 
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the cofactor expansion of \IJ\, we find 



on 



t-i 



1 



d 



I3\U\ aG(^'''') 



jj{s,s')^is.s') 



1 



(253) 



s=-l 



where {/^^'* denotes a cofactor of the (s, s') element of the 
matrix U, which does not include G'-"-""^. The (t + l) x (i + 1) 
matrix L ~ (L^'*''* then becomes 



1 



j(o,o)iy(o,-i) 



j(-i,-i)^(-i,t-i) ^^ 

j(0,0)^(0,t-l) 



+ 

/ j(t-i,t-i)^(t-i,-i) ... j(t-i,t~i)j^(t-i,t-i) \ 



V 

' j(o,o) . 











J(-l.-l) \ 

J(0,0) 

j(t-i,t-i) 



+ 



+ 



/ j(t-i,t-i) . . . j(t-\.t-\) \ 
••• 



A*adjt/T 



V ••• / 

- Jo «) (C/^)"^ - Ji ® A(J7^)-i 
Ji® A*(J7^)-\ 



(254) 



where TJ = /3f/ and adjA denotes an adjoint matrix of A. 
The definition of is given by ( |109l l. 

Due to Q = O, 51 can be expanded as ln|A + Q| = 
TrlnA + tr A^^Q. The Q^"'^') is obtained as g^"'^') = 
and we then have Q = -i^M^ = -i^(l + 
fiG)~^D{\^^G^)-^, where M = (/3-il+G)-i£)(/3-il + 
G^)-i and = /?£). Noting that |1 - /3B^| only 

contains fc'**) in a single row, fc'^'*^ is obtained as k^^^ = |A[^]|, 

where B = B\k=o = -J2t=-i J^'^^^G'^^'''' and the defini- 
tion of A[5] is given by (IIO8I 1. 

Some macroscopic parameters are found to vanish in the 
saddle-point: k^"^ = '^'^ = 0. In the case of the BP- 
base detector, the remaining macroscopic parameters can all 
be expressed in terms of three observables, i.e., the average 



overlap m'^^\ the average single-user correlation C'-'*'* and 
the average single-user response function G^**'* \ which are 
defined by (l44l l - (|46] |. The averaged generating functional 
Zltp] is dominated by a saddle-point in the large system limit. 
We then arrive at Proposition 2] 
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